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ABSTRACT 



The procedure and data of the Second Annual Award for Excellence in Teaching 
are discussed and studied. Basic statistics of balloting are presented and increased partici- 
pation is noted. The techniques of data reduction are described. Some special studies are 
made concerning (i) score data has distribution of mixed exponential type (ii) the effect 
on the score of whether or not an instructor teaches during the quarter of the balioting 
(iii) concordance determination with last year’s ranking of instructors. Recommendations 
for modifying future procedural items are made. 



TABLE OF CONTENTS 

ABSTRACT 2 

LIST OF TABLES 5 

LIST OF FIGURES 6 

I. INTRODUCTION 7 

II. THE BALLOT 9 

A. FORMAT 9 

B. THE BALLOTING PROCESS 10 

C. BALLOT VALIDATION 10 

1. Assembling the Data 10 

2. Transfering the Data to Tape 11 

III. STATISTICAL ANALYSES 12 

A. POPULATION CHARACTERISTICS 13 

B. CLOSER OBSERVATIONS i4 

C. DISTRIBUTION OF Mj 19 

D. DISTRIBUTION OF N k 20 

E. DETERMINATION OF S R 22 

F. DOMINANCE 

31 

1. Background 

32 

2. Development 

3. Results 33 

G. PAIRED COMPARISONS 35 

3 c i 

1. Development 

2. Results 33 

H. COMMENTS 36 



I. THE SELECTION 36 

IV. FURTHER ANALYSES 39 

A. BACKGROUND AND SUMMARY 39 

B. SCORE DISTRIBUTION STUDY 40 

C. TEACHING EFFECT 51 

D. CONCORDANCE ESTIMATION 54 

V. CONCLUSIONS 58 

APPENDIX A — SAMPLE BALLOT 59 

APPENDIX B - COMPUTER PRINTOUT 63 

LIST OF REFERENCES 88 

INITIAL DISTRIBUTION LIST 89 

FORM DD 1437 90 



I 



LIST OF TABLES 



I. BALLOT BREAKDOWN 

II. FACULTY 

III. BRANCH OF SERVICE 

IV. RANK 

V. VOTER CATEGORY 

VI. FREQUENCY TABLE OF N k 

VII. SCORSORT SUMMARY 

VIII. DOMINANCE TESTS RESULTS 

IX. PAIRED COMPARISONS 

X. CHI SQUARE TEST DATA FY 70, ALL 

XI. CHI SQUARE TEST DATA FY 70, Q3, TEACHING 

XII. CHI SQUARE TEST DATA FY 70, Q3, NOT TEACHING .... 

XIII. CHI SQUARE TEST DATA FY 70, Q4, TEACHING 

XIV. CHI SQUARE TEST DATA FY 70, Q4, NOT TEACHING . . . . 

XV. CHI SQUARE TEST DATA FY 71, Q3, ALL 

XVI. CHI SQUARE TEST DATA FY 71, Q3, TEACHING 

XVII. CHI SQUARE TEST DATA FY 71, Q3, NOT TEACHING . . . . 

A 

XVIII. A VALUES 

XIX. p VALUES 

XX. p VALUES FOR X 2 / Al 

XXI. p VALUES FOR q TEST 

XXII. CONCORDANCE MATRIX 



13 

15 

16 

17 

18 
21 
11 
34 
37 

42 

43 

44 

45 

46 

47 

48 

49 

50 

51 

52 

53 
55 



LIST OF FIGURES 



1. RECOMMENDED PART A 19 

2. HISTOGRAM OF Mj 20 

3. HISTOGRAM OF N k 21 

4. SCATTER DIAGRAM OF Z k vs N k 30 

5. DOMINANCE PLOT 33 

6. HISTOGRAM OF S k 41 



I. INTRODUCTION 



The Superintendent of the U. S. Naval Postgraduate School, Rear Admiral 
Robert W. McNitt, USN, directed in May of 1969, that an award be established to 
recognize excellence in teaching. To comply with his directives a committee was 
formed from among the faculty. The objective of this committee was to select a 
recipient each year for the Outstanding Teaching Award to be given during the June 
graduation. 

This committee was to be composed of a Chairman, a secretary and five members. 
These seven people were to be chosen to provide a wide variety of backgrounds and 
curricula. Each year four persons would leave the committee and be replaced by four 
new members. Thus, new ideas and concepts would be engendered in the selection 
process while retaining a core of persons familiar with the existing system. 

The members of this year’s committee were Professors Kovach (Chairman), 
Hoisington (Vice-Chairman and Secretary), Musgrave, Nunn, Renard, Read and Schultz. 
The committee decided to retain from last year’s committee the same philosophical 
outlook concerning the nature of excellence in teaching along with how it should be 
reliably identified and evaluated. The interested reader will find this outlook in 
Robert William Geary’s Thesis, The First Annual Award f or Excellence in Teaching, 

A Study of Procedure and Analysis of Data, June 1970. However, to set the ground- 
work for this paper certain guidelines for the selection process will be discussed. These 
guidelines are as follows: 

1. All faculty with the exception of the Department Chairmen and the members 
of the selection committee are eligible to receive the award. 

2. The voting population includes all current students, faculty. Curricular 
Officers and Assistant Curricular Officers. Also included this year is the 
group of Naval Officers which graduated in 1969. 

3. Participation in the poll is entirely voluntary and no one is required to 
sign his ballot. 
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4. Care and discretion are to be used to ensure that no ranking of candidates 
is published. 

5. For the ballot to be valid the voter must indicate himself to be reasonably 
well acquainted with at least five members of the candidate list. 

6. The voting population is cautioned not to vote on the basis of personality 
or popularity but on teaching ability and performance. 

In order to maintain total objectivity among the selection committee it was 
determined that an individual from outside the faculty would handle the ballots, 
collate the data and submit the results to the selection committee. Only after the 
recipient of the award had been selected, would the winner’s name be made known 
to the selection committee. 
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II. TIIE BALLOT 



A. FORMAT 

An instruction sheet was attached to the front of the ballot to explain the purpose 
of the award, give instructions on how to fill out the ballot, and explain the require- 
ments for validating the ballot. Results indicate that greater care should be exercised 
in preparing the cover sheet, particularly in the area of ballot validity. This is covered 
in greater length later in this section. 

The ballot itself was composed of four parts. Part A was a statistical gathering 
aid to help in an analysis of the voting population characteristics. The voter was 
asked to identify his voter category, rank, Branch of Service, and Curricular area if 
military; if civilian his academic rank and department were to be ended. 

Part B included three blanks to be used in indicating first, second, and third 
choices for the award using the four digit identification number in Part C. 

Part C of the ballot identified the candidates for the voter. Each eligible faculty 
member was listed in alphabetical order under the heading of his department. A four 
digit identification number preceded each candidate. The voter was asked to circle 
this identification number of all those faculty members with whom he considered 
himself to be reasonably well acquainted in an academic capacity. 

The last portion of the ballot, Part D, was to be used to comment on the reasons 
for his selection. This portion was not mandatory, but was considered beneficial in 
the final selection processes. 

The major variations in this year’s ballot from the year before were as follows: 

1 The second portion of the ballot (the blanks) was added to provide a 
faster and safer means of extracting the nominations from the ballot. 

2. The blanks were removed from the left side of the four-digit identification 
number. 
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The only changes recommended for the actual ballot for next year would be to 
include another blank in Part B that can be marked with a “1” if a comment is made 
in Part D and to assign each teacher a permanent four digit identification number for 
Part C instead of renumbering each year. This would further facilitate data retrieval. 
Included as Annex A of this document is a FY ’71 ballot. 

B. THE BALLOTING PROCESS 

The method whereby the ballots were distributed and returned merits attention, 
for the selection process is only as good as the response of the voters. 

Last year the ballots were distributed through the Curricular Officers and Section 
Leaders. In an attempt to increase student participation the selection committee 
decided a new method should be found for distributing and returning the ballots. 

The ballots were distributed utilizing the Student Mail Center (SMC) this year. The 
completed ballot was placed in an envelope addressed to the Superintendent of the 
Naval Postgraduate School and deposited in a marked box outside the SMC. The 
envelopes were coded 55 BAL to help route them to the person assembling the data. 

The Alumni selected to participate were the Naval Officers who graduated in 
calendar year *69. Their names and addresses were taken from a data tape maintained 
in the computer facility after the addresses had been updated [ref. 4] . The completed 
ballots were returned to the mailing address previously mentioned. 

C. BALLOT VALIDATION 
1. Assembling the Data 

Once the completed ballots had been returned to a central location, the 
validation process for each ballot began. The ballots were removed from their envelopes 
and cover sheets were discarded. 

The validation process encompassed counting to ensure that Part C of the 
ballot had at least five identification numbers circled and that Part B did not l ave 
any write-in nominations. If less than five identification numbers were circied in 
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Part C or if the only nominations were write-ins, the ballot was considered void and 
placed in a separate stack. Valid ballots were numbered so that specific data could 

be retrieved at a later date if necessary. 

The chief cause of void ballots was the restriction that at least five persons 

be known by the voter. Of the 281 void ballots, 265 were deemed void for this 
reason. 



Another fifteen ballots or 5.7% of all void ballots were deemed void 
because write-in nominations of ineligible faculty had been made in place of numbers 
from the eligible faculty. 

One reason for the high number of void ballots, 281 out of 1722, could be 
attributed in part to the influx of first quarter students. Each person receiving a 
ballot was requested to return the ballot whether or not he knew five eligible faculty 

members. Yet, the number of void ballots is too great to be the result of first quarter 

„ • , _ ^1.. A f ,A thot o IqvrfO mi mhpr of n^rsnns misinter- 

students alone, lhereioie, it was conv. lW uv.J - i-n - a 

preted the instructions. 



2. Transfering the Data to Tape 

When the above validation process was completed, the ballots were given 
to key punch operators to transfer the data to IBM cards. The IBM cards were used 
as data for a COBOL routine to check for key punch errors and transfer virtually 
error free data to a magnetic tape. 

An explanation of how the COBOL routine checked for key punch errors 
will also clarify why the identification numbers were used. The four digits comprising 
the teacher identification number were arranged so that the first three digits identified 
the particular teacher. The first teacher on the ballot had an identification number 
101. One was added to this number to form the identification number for the second 
teacher, 102. The process was repeated until the last identification number was formed. 
The first three numbers formed a vector, N, which when multiplied by a fixed 
weighting vector, S , formed an inner product. The last digit of this inner product 
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formed the fourth digit of the identification number. As an example teacher 299 would 



yield 



(299) 




= 6 + 45 + 9 = 60. 



Thus, his full number would be 2990. 

The above described procedure proved very useful for only correct identifi- 
cation numbers were transferred from the ballot to the tape. An error in key punching 
would cause an error message giving the ballot number and its cause for rejection. 
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III. STATISTICAL ANALYSES 



A. POPULATION CHARACTERISTICS 

One of the most important factors of the voting was the voter response. After 
sorting the valid from void ballots the percentages of response were determined for 
each voter cagegory. Table I below lists findings from the breakdown of the ballots. 



Table I. BALLOT BREAKDOWN 





No. Eligible 


No. Valid 


% Valid 


% Increase Over 


Voter Category 


Voters 


Ballots 


From Eligible 


Previous Year 


Student 


1890 


1023 


54.1 


5.1 


Alumni 


525 


246 


46.9 


7.9 


Faculty 


314 


107 


34.1 


8.5 


Curricular Officer 


16 


5 


31.3 


(1.7) 


Unknown* 


— 


60 







* Includes 40 ballots in which Part A was omitted 
() indicates negative number 



The last column of Table I presents the percentage change over the last year, indicating 
that all valid voter categories, except that of the Curricular Officers 1 , improved. 

The number of eligible voters in both the student and faculty categories increased 
appreciably; therefore, the percentage increases shown indicate a significant increase 
in response. 

This year before mailing the alumni ballots, the address tape maintained by 
the computer center was updated. Even though five less ballots were mailed the 
increase in response was 7.9 percent. This supports the belief that updating of 

the tapes would improve alumni response. 

The large number of ballots with no voter category indicated this to be a 
definite problem area. Two possible causes are listed on the following page: 

1 Curricular Officers who identified themselves as alumni had their ballots included 
in the alumni group. 
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1 The instructions concerning the voter category were vague and misinter- 
preted. 

2. The voter did not want to take the time to fill out Part A. 

3. The voter did not want to be identified. 

Of the above the most likely explanation concerns the instruction sheet. The reference 
to Part A was given under Step O. This may have led the voter to assume that no 
importance was placed on this information. However, in order to improve the 
balloting procedure, information concerning the voters is necessary. The format of 
Part A should be changed to place the voter category under the heading of VOTER 
CATEGORY instead of beside the heading in hopes that the voters’ attention will be 

called to the choices. 

The 1441 valid ballots consisted of 71 percent students, 17.1 percent alumni, 

7.4 percent faculty, 0.3 percent Curricular Officers and 4.2 percent unknown. The 
420 ballot increase over last year reflects both an increase in voter population and 
improved participation in the voting. 

B. CLOSER OBSERVATIONS 

After studying the voter categories it was determined to research the military 
and faculty data for further information concerning the voter population. Table II 
presents a breakdown of faculty voters by curricular department and academic rank. 
The columns indicate the different ranks and rows indicate academic department. 

By scanning the j th column and i th row, a determination of the number of valid 
ballots from faculty of i department and of j rank can be made. 

Tables III, IV and V breakdown the military voters by Branch of Service, Rank 
and Voter Category respectively in each curricular area. The i th row gives the 
curricular area and the j th column gives Branch of Service, Rank, or Voter Category 
according to the title of the table. The “faculty” column in Table V, Voter Category, 
contains those ballots marked faculty and military. One possible explanation for a 
person being listed as military and a member of the faculty would be that the voter 
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Table II. FACULTY 



Rank 

Department 


Instructor 


Assistant 

Professor 


Associate 

Professor 


Professor 


Total 

Voted 


Number 

Enrolled 




Aeronautics 






4 


3 


7 


22 


Aviation 

Safety 


1 


1 






2 


5 ! 


Business Administra- 
tion & Economics 


5 


2 


2 


4 


13 


31 


Electronics 
1 Engineering 




3 


8 


5 

1 


16 

“T 


45 


Government 
& Humanities 


1 




3 


1 


1 

5 


11 


Material Science 
& Chemistry 






2 


3 


5 


13 


Math 


3 


2 


4 


2 


11 


40 


Mechanical 

Engineering 




2 


2 


3 


7 


15 




Meteorology 




3 


2 


2 


7 


15 




Oceanography 


1 


1 


1 


3 


6 


15 




Operations 
A na 1 


2 


1 


8 


1 


12 


46 




r\ 1 \CK 1 V 3 1 D 

Physics 


1 


2 


2 


8 


13 


35 




Navy Management 
Systems Center 






1 


2 


3 


21 
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Table III. BRANCH OF SERVICE 



CURRICULAR 

AREA 


NAVY 


ARMY f> 


4ARINE 


COAST 

GUARD 


AIR , 

FORCE 


r OREIGN 


No Curriculum Noted 


16 




1 






1 


30 

Operations 

Analvsis 


123 


26 


18 


1 




7 


31 

Aeronautical 

Engineering 


68 




1 






1 


32 

Electronics & 
Communication 


119 




1 


1 




10 


33 

Ordnance 


66 


7 

1 

1 


2 




1 

i 


3 

1 


34 

Naval 

Engineering 


52 










2 


35 

Environmental 

Science 


no 










2 


36 

Management & 
Computer Science 


230 




9 


16 




3 


37 

Engineering 

Science 


35 


1 






1 




38 

Baccalaureate 


134 












39 

Defense 

N 1 1 m nr> rn pn t 


7 






1 




4 


ivianagemeu i 
TOTALS 


960 


34 


32 


19 


2 


33 
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Table IV. RANK 



CURRICULAR 

AREA 


01 


02 


03 


04 


05 


06 


No Curriculum Noted 


3 


2 


10 


5 


1 


1 


30 

Operations 

Analysis 


29 


9 


88 


63 


6 1 


2 


31 

Aeronautical 

Engineering 


14 


2 


40 


21 


4 




32 

Electronics & 
Communication 


30 


6 


69 


41 


4 


3 


33 

Ordnance 

| 


13 


9 


33 

1 


26 

1 


4 


1 


34 

Naval 

Engineering 


9 


10 


31 


11 


1 


2 


35 

Environmental 

Science 


20 


10 


51 


39 


4 




36 

Management & 
Computer Science 


68 


12 


64 


86 


62 


9 


37 

Engineering 

Science 


5 


1 


20 


10 


2 




38 

Baccalaureate 


32 


1 


39 


53 


37 




39 

Defense 

^ m riA m pnf 


1 






3 


4 


5 


IVIcina^cl I ICII l 

TOTALS 


224 


62 


445 

_L 


358 


128 


23 
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Table V. VOTER CATEGORY 



CURRICULAR 


No 


STUDENTS 


Faculty 


Alumni 


Curricular 


AREA 


Voter 

Category 


Enroll- 

ment 


Votes 


Per- 

centage 


Officers 


No Curriculum Noted 






19 




1 


5 




30 

Operations 

Analysis 




291 


182 


62.5 


1 


24 




31 

Aeronautical 

Engineering 




209 


62 


56.9 




21 




32 

Electronics & 
Communication 




238 


134 


56.3 




20 




33 

Ordnance 




94 


70 


74.5 


1 


21 


1 


34 

Naval 

Engineering 




90 


48 


53.3 




1 

17 


1 


35 

Environmental 

Science 




170 


97 


57.0 


3 


24 


2 


36 

Management & 
Computer Science 


2 


390 


227 


58.2 


3 


77 


2 


37 

Engineering 

Science 




115 


37 


32.2 




5 




38 

Baccalaureate 




307 


132 


42.9 


1 


32 




39 

Defense 

Management 


1 


86 


15 


17.4 








TOTALS 


3 


1890 


1023 

I 


54.1 


10 


246 


5 
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was one of the school’s Navy Instructor personnel. Another possible explanation 
would be an error in completing Part A of the ballot. However, the first explanation 
was deemed to be the more probable of the two. Figure 1, Recommended Part A, 
is a possible solution to some of the previously mentioned problems in this area. 



Voter Category 











S Student 
F Faculty 
A Alumnus 
















C Curricular Officer 






Students, 


Alumni, Curricular Officers 


Faculty 




Rank 


Branch of Service 


Curricular Area Rank 


Department 


01 


N 


Navy 


30 


Ops Analysis IR 


Instructor 


AE 


Aeronautics 


02 


M 


Marine Corps 


31 


Aero Engr AT 


Asst Prof 


AO 


Aviation Safety 


03 


A 


Army 


32 


Elec <& Comm Engr AC 


Assoc Prof 


MN 


Bus Ad & Econ 


04 


C 


Coast Guard 


33 


Ordnance PR 


Professor 


EE 


Elec Engr 


05 


L 


Ait Force 


34 


Naval Engr 




GH 


Govt & 


06 


F 


Foreign 


35 


Envir Sci 






Humanities 


07 






36 


Mange & Comptr Sci 




MC 


Mat Sc & Che in 








37 


Eng Sci 




MA 


Math 








38 


Baccalaureate 




M C 

MIL 


M c c h Engr 








39 


Def Management 




MR 


Meteorology 














OC 


Oceanography 














OA 


Ops Analysis 














PH 


Physics 














NS 


Navy Mgt Sys 



Center 



Figure 1. RECOMMENDED PART A 
C. DISTRIBUTION OF Mj 

An interesting statistic concerning the voting population was the number of 

eligible faculty members known by the i th voter. As the i th voter becomes acquainted 

» ' 

with more faculty, he should be able to choose better for he is more familiar with 
the teaching methods of the eligible population. 

Figure 2 presents a histogram of Mj, the number of eligible faculty members 
known by the i*'* 1 voter. The mean or average number of faculty members known by 
the voting population was 16.8. At first glance this appears to be a good number, 
then there was the realization that a ballot must have hade at least five eligible faculty 



19 



members marked known before it was considered to be valid. This fact along with a 
standard deviation of 9.7 showed that it was not safe to assume the average voter’s 
choice to be founded on a large base of experience. Until such time as a method 
can be devised to weight the ballots based on the voter’s experience, it should suffice 
to say that considering the wide range of voter experience (1 to 8 quarters) a large 
varience, 94.7, in Mj was and can be expected. 




D. DISTRIBUTION OF N k 

Now that an indication of how many faculty were known on each ballot has been 
given, it would prove useful to discuss how many ballots indicated acquaintance with 
each faculty member. Figure 3 gives a histogram of N k , the number of voters 
identifying teacher k. Table VI presents the frequency table used for this discussion. 
The mean number of voters who know teacher k was 82.9 with a standard deviation 
of 62.5. Of special interest was the fact that 514 people indicated they knew one 
teacher. Due to this phenomena the teacher would have had to have significantly 
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more votes than the others in order to place high in the standings. This pointed out 
one of the major weaknesses of the present system of selection which is expounded 
upon in subsection E Determination of S k . 




FREQUENCY TABLE OF N k 



Table VI 

CELL 
1 - 24 
26 - 50 
51 - 75 
76-100 
101-125 
126-150 
151-175 
176-200 
201-225 
226-250 
251-275 
276-300 
301-325 

501-515 



FREQUENCY 

22 

64 

56 

61 

44 

16 

11 

6 

5 

3 

0 

1 

0 

1 
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E. DETERMINATION OF S k 

After the balloting data had been transfered to magnetic tape, a series of computer 
programs were run to determine specific statistics from the data. Appendix B contains 
a listing of these computer programs. One of the first important statistics determined 
was the score, S k , of each professor. In order to perform this exercise the data was 
transferred to magnetic tape by the use of the computer routine “TRANSFER”. A 
second computer routine “PROFSORT” was run to rearrange the data from ballot 
format to a format identifying each teacher, T k . That routine also gave a frequency 
distribution of Mj. After the “PROFSORT” routine had been completed, the third 
routine, “SCORSORT” was run giving a printout of each teacher and his respective 

score which had been determined as follows: 

l Teacher k’s first, second, and third place votes Xj, x 2 and x 3 respectively 
were divided by the number of persons identifying teacher k in Part C of 
the ballot. This yielded a vector V k . 

2. The vector V k was multiplied by a weighting vector W and the product 
yielded S k , the score for teacher k. Mathematically this operation was 
expressed as 

S k - V k • W 

The following table presents a summary of the “SCORSORT” printout showing 
k, S k , Z k , N k , X lk , X 2k , and X 3k respectively. 



Table VII. SCORSORT SUMMARY 



k 


s k 


Nk 


1 


2.12121 


66 


2 


2.08955 


134 


3 


1.84000 


25 


4 


1.82692 


104 


5 


1.77419 


31 


6 


1.69474 


95 


7 


1.68831 


77 


8 


1.63134 


217 


9 


1.59420 


69 


10 


1.55556 


99 


11 


1.53704 


54 



Z k 


X 1k 


X 2k 


140 


* 26 


16 


280 


59 


18 


46 


8 


6 


190 


38 


14 


55 


11 


5 


161 


29 


18 


130 


27 


9 


354 


58 


47 


110 


20 


12 


154 


25 


22 


83 


15 


• 8 



x 3k 

4 

8 

2 

10 

1 

9 

4 

28 

6 

10 

7 
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k 

12 

13 

14 

15 

16 

17 

18 

19 

20 

21 

22 

23 

24 

25 

26 

27 

28 

29 

30 

31 

32 

33 

34 

35 

36 

37 

38 

39 

40 

41 

42 

43 

44 

45 

46 

47 

48 

49 

50 

51 

52 

53 

54 

55 



Table VII. SCORSORT SUMMARY (continued) 

S k N k Z k X lk X 2k X 3k 



1.50000 


44 


66 


1.47312 


93 


137 


1.42857 


49 


70 


1.41935 


93 


132 


1.28571 


28 


36 


1.27907 


129 


165 


1.27149 


221 


281 


1.24528 


159 


198 


1.18103 


116 


137 


1.09917 


121 


133 


1.08929 


56 


61 


1.08696 


115 


125 


1.08197 


61 


66 


1.07200 


125 


134 


1.02083 


96 


98 


0.99415 


171 


170 


0.98718 


78 


77 


0.98077 


52 


51 


0.95652 


46 


44 


n .94737 


57 


54 


0.91667 


96 


88 


0.91667 


24 


22 


0.89011 


91 


81 


0.86207 


29 


25 


0.82836 


134 


111 


0.80488 


82 


66 


0.79762 


84 


67 


0.78571 


28 


22 


0.77193 


57 


44 


0.72436 


156 


113 


0.70476 


105 


74 


0.70000 


100 


70 


0.70000 


20 


14 


0.68889 


45 


31 


0.68889 


45 


31 


0.68750 


32 


22 


0.68182 


110 


75 


0.67568 


74 


50 


0.66990 


103 


69 


0.66000 


50 


33 


0.65049 


103 


67 


0.64444 


90 


58 


0.63889 


36 


23 


0.63415 


82 


52 
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65 
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67 
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Table VII. SCORSORT SUMMARY (Continued) 



S k 


N k 


Z k 


X 1k 


X 2k 


X 3k 


0.62821 


78 


49 


9 


2 


9 


0.60938 


64 


39 


8 


3 


1 


0.58511 


94 


55 


8 


8 


7 


0.57798 


109 


63 


7 


12 


11 


0.57463 


134 


77 


16 


4 


5 


0.57317 


82 


47 


6 


9 


5 


0.57143 


77 


44 


6 


8 


4 


0.56522 


184 


104 


14 


20 


8 


0.56436 


101 


57 


8 


9 


7 


0.56383 


94 


53 


7 


7 


11 


0.56250 


48 


27 


5 


1 


5 


0.56000 


50 


28 


4 


4 


4 


0.55670 


97 


54 


8 


3 


6 


0.54369 


103 


56 


10 


3 


10 


0.53226 


62 


33 


4 


6 


5 


0.51961 


102 


53 


6 


13 


3 


0.51948 


77 


40 


6 


5 


6 


0.51515 


33 


17 


1 


5 


3 


0.51316 


76 


39 


4 


9 


5 


0.51304 


115 


59 


8 


8 


11 


0.50769 


65 


33 


4 


6 


5 


0.50549 


91 


46 


7 


6 


6 


0.50000 


104 


52 


9 


6 


4 


0.48837 


43 


21 


3 


4 


1 


0.48780 


82 


40 


7 


3 


6 


0.47500 


40 


19 


2 


4 


3 


0.47287 


129 


61 


9 


9 


7 


0.47170 


53 


25 


2 


6 


5 


0.46087 


115 


53 


8 


7 


7 


0.46067 


89 


41 


5 


7 


7 


0.45455 


44 


20 


3 


3 


2 


0.44828 


29 


13 


2 


1 


3 


0.43662 


142 


62 


8 


12 


6 


0.42991 


107 


46 


7 


7 


4 


0.42391 


92 


39 


5 


6 


7 


0.42308 


52 


22 


3 


3 


4 


0.41758 


91 


38 


6 


4 


6 


0.41045 


134 


55 


5 


11 


13 


0.40909 


44 


18 


2 


2 


6 


0.40288 


139 


56 


6 


11 


10 


0.39286 


56 


22 


4 


1 


4 


0.39286 


56 


22 


3 


3 


4 


0.39024 


41 


16 


2 


3 


2 
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111 
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123 
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Table VII. SCORSORT SUMMARY (continued) 



Sk 


N k 


Z k 


x 1k 


X 2k 


X 3k 


0.38710 


124 


48 


6 


8 


8 


0.37975 


79 


30 


3 


6 


6 


0.36364 


77 


28 


5 


2 


4 


0.36232 


69 


25 


4 


2 


5 


0.36184 


152 


55 


6 


12 


7 


0.35928 


167 


60 


4 


11 


22 


0.35821 


67 


24 


4 


3 


2 


0.35135 


111 


39 


3 


10 


7 


0.35099 


151 


53 


8 


7 


7 


0.34286 


140 


48 


6 


9 


6 


0.33696 


92 


31 


5 


4 


3 


0.33333 


117 


39 


2 


9 


13 


0.33333 


36 


12 


2 


2 


0 


0.33333 


33 


11 


2 


0 


3 


0.33333 


30 


10 


1 


1 


4 


0.33000 


100 


33 


3 


6 


9 


0.32432 


111 


36 


5 


5 


6 


0.31731 


104 


33 


5 


3 


7 


0.31707 


41 


13 


2 


1 


3 


0.31690 


142 


45 


6 


8 


5 


0.31646 


79 


25 


2 


6 


5 


0.30909 


110 


34 


5 


6 


2 


0.30769 


65 


20 


0 


7 


6 


0.30081 


123 


37 


7 


3 


3 


0.29412 


68 


20 


• 1 


5 


6 


0.29412 


17 


5 


0 


2 


1 


0.28571 


35 


10 


2 


1 


0 


0.28523 


298 


85 


11 


15 


11 


0.28205 


39 


11 


2 


1 


1 


0.27922 


154 


43 


7 


6 


3 


0.27692 


65 


18 


3 


2 


2 


0.27500 


40 


11 


2 


0 


3 


0.27027 


185 


50 


v 8 


5 


8 


0.25806 


62 


16 


2 


4 


0 


0.25641 


39 


10 


1 


2 


2 


0.25352 


71 


18 


2 


3 


4 


0.24390 


41 


10 


1 


2 


2 


0.24324 


37 


9 


1 


2 


1 


0.24227 


194 


47 


5 


10 


7 


0.23256 


86 


20 


2 


3 


6 


0.22951 


61 


14 


1 


2 


6 


0.22727 


66 


15 


0 


6 


3 


0.22727 


22 


5 


1 


0 


1 
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Table VII. SCORSORT SUMMARY (continued) 



k 


s k 


N k 


142 


0.22642 


53 


143 


0.22500 


40 


144 


0.22222 


162 


145 


0.22000 


100 


146 


0.21875 


64 


147 


0.21698 


212 


148 


0.21384 


159 


149 


0.21053 


114 


150 


0.20988 


81 


151 


0.20952 


210 


152 


0.20588 


68 


153 


0.20561 


107 


154 


0.20000 


80 


155 


0.19863 


146 


156 


0.19780 


91 


157 


0.19091 


110 


158 


0.18288 


514 


159 


0.18000 


100 
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0.17647 


51 
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0.17143 


1 05 


162 


0.17045 


88 
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83 
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0.16783 


143 
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0.16667 


60 
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36 
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0.16250 


80 
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0.16250 


80 
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0.15337 


163 
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0.15126 
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0.14894 


47 
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27 
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0.14783 


115 
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62 
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0.14493 


69 
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0.14458 


83 
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0.13861 


101 
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0.13580 


81 
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0.13433 


67 
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0.13333 


30 
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0.12903 


62 
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0.12821 
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0.12821 


39 


184 


0.12745 


102 



1 3 2 

0 3 3 
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0 6 10 

2 2 2 

5 10 6 

1 10 10 

4 3 2 
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2 3 8 
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1 4 6 

0 6 9 

10 19 16 

1 2 8 

2 0 1 

3 2 2 

2 3 1 

1 4 2 

1 7 6 

2 0 2 
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1 2 5 
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0 2 0 
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2 2 0 

1 2 6 

2 1 1 
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0 1 3 
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34 
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22 
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18 

9 
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Table VII. SCORSORT SUMMARY (continued) 



Sk 


N k 


Z k 


X 1k 


X 2k 


x 3k 


0.12727 


55 


7 


1 


0 


3 


0.12658 


79 


10 


1 


2 


2 


0.12381 


105 


13 


1 


2 


5 


0.12245 


49 


6 


1 


1 


0 


0.11905 


42 


5 


1 


0 


1 


0.11864 


118 


14 


2 


2 


2 


0.11538 


52 


6 


1 


0 


2 


0.11483 


209 


24 


3 


4 


4 


0.11215 


107 


12 


2 


1 


2 


0.11009 


109 


12 


1 


2 


4 


0.10784 


102 


11 


0 


4 


3 


0.10588 


85 


9 


1 


1 


3 


0.10526 


76 


8 


1 


1 


2 


0.10448 


67 


7 


1 


1 


1 


0.10345 


58 


6 


1 


0 


2 


0.10345 


58 


6 


1 


1 


0 


0.10300 


233 


24 


4 


3 


2 


0.10227 


88 


9 


0 


4 


1 


0.10145 


69 


7 


0 


3 


1 


0.10112 


89 


9 


0 


4 


1 


0.10000 


30 


3 


0 


1 


1 

1 


0.10000 


10 


1 


0 


0 


1 


0.09756 


41 


4 


1 


0 


0 


0.09655 


145 


14 


1 


3 


4 


0.08738 


103 


9 


1 


1 


3 


0.08333 


180 


15 


0 


5 


5 


0.08333 


72 


6 


0 


2 


2 


0.08333 


24 


2 


0 


1 


0 


0.08333 


24 


2 


0 


1 


0 


0.08163 


49 


4 


0 


1 


2 


0.08000 


50 


4 


1 


0 


0 


0.07843 


51 


4 


0 


2 


0 


0.07813 


64 


5 


1 


0 


1 


0.07692 


52 


4 


0 


1 


2 


0.07563 


119 


9 


1 


2 


1 


0.07547 


53 


4 


1 


0 


0 


0.07500 


80 


6 


1 


0 


2 


0.07407 


108 


8 


1 


2 


0 


0.07143 


14 


1 


0 


0 


1 


0.07092 


141 


10 


0 


3 


4 


0.06993 


143 


10 


0 


2 


6 


0.06944 


72 


5 


0 


1 


3 


0.06849 


73 


5 


0 


1 


3 
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Table VII. SCORSORT SUMMARY (continued) 



k 


Sk 


N k 


Z k 


x 1k 


X 2k 


X 3k 


228 


0.06667 


15 


1 


0 


0 


1 


229 


0.05941 


101 


6 


0 


2 


2 


230 


0.05195 


77 


4 


0 


2 


0 


231 


0.05000 


40 


2 


0 


1 


0 


232 


0.05000 


20 


1 


0 


0 


1 


233 


0.04977 


221 


11 


2 


0 


3 


234 


0.04918 


183 


9 


0 


3 


3 


235 


0.04444 


45 


2 


0 


1 


0 


236 


0.04054 


74 


3 


0 


1 


1 


237 


0.04000 


50 


2 


0 


0 


2 


238 


0.03896 


77 


3 


0 


0 


3 


239 


0.03846 


26 


1 


0 


0 


1 


240 


0.03704 


81 


3 


0 


1 


1 


241 


0.03636 


55 


2 


0 


1 


0 


242 


0.03265 


245 


8 


0 


1 


6 


243 


0.03175 


63 


2 


0 


0 


2 


244 


0.03125 


64 


2 


0 


1 


0 


245 


0.02632 


38 


1 


0 


0 


1 


246 


0.02542 


118 


• 3 


0 


1 


1 


247 


0.02500 


40 


1 

1 


0 


r\ 

KJ 


1 

1 


248 


0.02353 


85 


2 


0 


1 


0 


249 


0.02174 


92 


2 


0 


0 


2 


250 


0.02105 


95 


2 


0 


1 


0 


251 


0.02000 


50 


1 


0 


0 


1 


252 


0.01563 


64 


1 


0 


0 


1 


253 


0.01493 


67 


1 


0 


0 


1 


254 


0.01136 


88 


i 


0 


0 


1 


255 


0.01111 


90 


1 


0 


0 


1 


256 


0.01053 


95 


1 


0 


0 


1 


257 


0.00909 


110 


1 


0 


0 


1 


258 


0.00000 


244 


0 


0 


0 


0 


259 


0.00000 


123 


0 


0 


0 


0 


260 


0.00000 


97 


0 


0 


0 


0 


261 


0.00000 


63 


0 


» 0 


0 


0 


262 


0.00000 


56 


0 


0 


0 


0 


263 


0.00000 


55 


0 


0 


0 


0 


264 


0.00000 


55 


0 


0 


0 


0 


265 


0.00000 


50 


0 


0 


0 


0 


266 


0.00000 


50 


0 


0 


0 


0 


267 


0.00000 


50 


0 


0 


0 


0 


268 


0.00000 


47 


0 


0 


0 


0 


269 


0.00000 


44 


0 


0 


0 


0 


270 


0.00000 


41 


0 


0 


0 


0 
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Table VII. 


SCORSORT SUMMARY (concluded) 




k 


s k 


N k 


Z k 


Xik 


X 2k 


X 3k 


271 


0.00000 


39 


0 


0 


0 


0 


272 


0.00000 


35 


0 


0 


0 


0 


273 


0.00000 


34 


0 


0 


0 


0 


274 


0.00000 


32 


0 


0 


0 


0 


275 


0.00000 


31 


0 


0 


0 


0 


276 


0.00000 


29 


0 


0 


0 


0 


277 


0.00000 


29 


0 


0 


0 


0 


278 


0.00000 


28 


0 


0 


0 


0 


279 


0.00000 


27 


0 


0 


0 


0 


280 


0.00000 


23 


0 


0 


0 


0 


281 


0.00000 


23 


0 


0 


0 


0 


282 


0.00000 


20 


0 


0 


0 


0 


283 


0.00000 


19 


0 


0 


0 


0 


284 


0.00000 


18 


0 


0 


0 


0 


285 


0.00000 


14 


0 


0 


0 


0 


286 


0.00000 


14 


0 


0 


0 


0 


287 


0.00000 


9 


0 


0 


0 


0 


288 


0.00000 


6 


0 


0 


0 


0 


289 


0.00000 


6 


0 


0 


0 


0 


290 0.00000 2 

S k is related to another statistic, Z k , 


0 

defined by 


0 


0 


0 



Z k - x ik ' w 

It is easily recognized that 




An analysis was conducted of the relationship between Z k and N k , where Z k was 
the total points received by teacher k and N k was the number of voters familiar with 
teacher k. A scatter diagram, Figure 4, was constructed and it was concluded from 
this diagram that no relationship more sophisticated than Z k = S k W k was warranted. 



F. DOMINANCE 

The data from the score statistics was used to generate other statistics to be used 
for award recipient selection. A method was needed to reduce the number of eligible 
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recipients without resorting to subjective guesses. Two methods for doing this were 
absolute dominance and complete dominance. These two concepts are discussed below. 

1. Background 

Associated with each teacher k, T k , are statistics which give specific information 

about the individual. These include: 

= the number of i th place votes received by T k 
N k = Number of voters knowing T k 

v ik = X ik /N k = fraction on P lace votes awarded to T k from the 
population that knew him 

W = a weighting vector 

The following structure for a preferential region was suggested by Professor 
T. Gawain, Chairman of the first Selection Committee. It would seem logical that W 
would be constrained such that 

w-| W9 > wg > 0 

since a first place vote should be worth more than a second place vote which in turn should 
be worth more than a third place vote. Further, since weighting all place votes the same 
or giving one of the places a zero weight would be contrary to requesting three separate 
rankings, there must be greater restrictions on W. In fact, it would seem logical to have 

0 < a-i < w 2 /wj ^ b x <1 

(2) 

0 < a 2 < w 3 /w 2 < b 2 < 1 

where (aj, b x ) and (a 2 , b 2 ) define a preferential region interior to the unit square. The 
values of a lt a 2 , b x and b 2 would be chosen based on a determination of the smallest 
and largest ratios acceptable for the three rankings. This study used 

a l = a 2 = 

and 

bi — b2 — 2/3 

a s the acceptable parameters. 



and 
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2. Development 



It is desired to compare T k ’s score, S k ,with other teachers scores, S k ’. The 
main interest lies in all values of W such that 

s k ^ S k’ 



or 



w l v lk + w 2 V 2k + w l V 3k ^ w l V lk’ 



+ w 2 V 2k* + w 3 V 3k 



(3) 



for all 



K’ f K. 



rearranging (3) yields 

wi( v i k - v ik’) + w 2( V 2k' V 2k’) + w 3( V 3k' V 3k’) ^ °- (4) 

Now if all values of V ik - V ik ’ > 0, i = 1,2,3, then S k > S k ’ regardless of W and S k ’ 
can be eliminated from competition. Thus, if T k has a vector V k such that V ik > V ik , 
i = 1,2,3, then T k absolutely dominates T k ’. This form of dominance is absolute 

dominance. 

The next form of dominance eliminates all teachers who, in order to win, 
must have weighting systems that violate wj. _> w 2 wg. Divide (4) by w 2 and 
rearrange yielding 

w 3 v lk~ V lk’ ^1 V 2k~ V 2k’ if V 3k - V 3k ’ > 0, (5) 

w — ' V 3k -V 3k ’ w 2 " v 3k‘ V 3k’ ’ otherwise the sense of the 

1 inequality is reversed 



Since the V ik and Vj k ’ values are known, the dominance relationship is determined 
by the values of Wj, i = 1,2,3. For the appropriate values of wj, i = 1,2,3, T k is not 
dominated by T k ’. 

Let y = w 3 /w 2 and note that 0 < y 1. Let X = w-^/w 2 and note that 
1 < X Then (5) becomes 



y > 



V lk~ V lk* 

V 3k- V 3k> 



X - 



V 2k- V 2k’ 

V 3k‘ V 3k’ 



for V 3k - V 3k ’ > 0. 



Figure 5 gives a plot of y vs x. The shaded area indicates the feasible region 



for weights. 
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y 




Figure 5. DOMINANCE PLOT 



Now consider the boundary 



y = 



Vlk-Vlk 

V 3k- V 3k’ 



X + 



V 2k- V 2k 

V 3k- V 3k 



This is the slope intercept form of the equation for a line, 
that for Tj. to be completely dominated by Tu’ either: 



Analysing this indicates 



V lk ’-V lk 

1) rr 77 — 7 > 0 and y(l) > 1, for v 3k * V 3k’ > 0 

V 3k' V 3k 

2) lk - -7 < 0 and y(l)< 0, for V 3k - V 3k ’ <0 

V 3k' V 3k 

3 ) V 3k - V 3k = 0 and 

a) V lk - V lk , < 0 and < 1 

b ) v lk ' v lk’ = 0 and V 2k < V 2k 

If any of these obtain , then T k is completely dominated by T k ’ otherwise there is 
no decision. T k will be eliminated if he is completely dominated by at least one T k ’. 
3. Results 

A computer routine was prepared based on the above theory and dominance 
tests were run. Table VIII presents a summary of the data used and the results of the 
dominance tests. The AD column lists the results of the Abolute Dominance tests 
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employing a 1 to indicate T k is absolutely dominated and a 0 to indicate it is not 
absolutely dominated. The CD column lists the results of the Complete Dominance 
tests using 1 and 0 as above to indicate complete domination. 



Table VIII. DOMINANCE TESTS RESULTS 



T k 


V 1k 


CM 

> 


1 


0.394 


0.242 


2 


0.440 


0.134 


3 


0.320 


0.240 


4 


0.365 


0.135 


5 


0.355 


0.161 


6 


0.305 


0.189 


7 


0.351 


0.117 


8 


0.267 


0.217 


9 


0.290 


0.174 


10 


0.253 


0.222 


11 


0.278 


0.148 


12 


0.273 


0.114 


13 


0.290 


0.118 


14 


C.224 


0.204 


15 


0.226 


0.204 


16 


0.214 


0.143 


17 


0.240 


0.116 


18 


0.222 


0.140 


19 


0.189 


0.189 


20 


0.216 


0.138 


21 


0.182 


0.089 


22 


0.214 


0.107 


23 


0.200 


0.096 


24 


0.197 


0.098 


25 


0.160 


0.176 


26 


0.198 


0.063 


27 


0.146 


0.146 


28 


0.179 


0.090 


29 


0.154 


0.154 


30 


0.174 


0.109 


31 


0.175 


0.070 


32 


0.146 


0.115 


33 


0.125 


0.167 


34 


0.154 


0.088 


35 


0.172 


0.069 


36 


0.112 


0.142 


37 


0.098 


0.159 



V 3R AD CD 

0.061 0 0 

0.060 0 0 

0.080 0 1 

0.096 0 1 

0.032 1 1 

0.095 0 1 

0.052 1 1 

0.129 0 1 

0.087 1 1 

0.101 0 1 

0.130 0 1 

0.182 0 1 

0.075 1 1 

0 1 22 1 1 

0.108 1 1 

0.143 0 1 

0.085 1 1 

0.104 1 1 

0:113 1 1 

0.043 1 1 

0.174 1 1 

0.018 1 1 

0.096 1 1 

0.098 1 1 

0.080 1 1 

0.104 1 1 

0.117 1 1 

o!o9o 1 1 

0.058 1 1 

0.043 1 1 

0.105 1 1 

0.104 1 1 

0.083 1 1 

0.099 1 1 

0.034 1 1 

0.097 1 1 

0.098 1 1 
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Table VIII. DOMINANCE TESTS RESULTS (concluded) 



T k 


V 1k 


V 2k 


V 3k 


AD 


CD 


38 


0.083 


0.179 


0.107 


1 


1 


39 


0.071 


0.143 


0.214 


0 


1 


40 


0.158 


0.070 


0.000 


1 


1 


41 


0.109 


0.103 


0.083 


1 


1 


42 


0.143 


0.057 


0.019 


1 


1 


43 


0.120 


0.090 


0.040 


1 


1 


44 


0.000 


0.250 


0.200 


0 


1 


45 


0.044 


0.222 


0.067 


1 


1 


46 


0.089 


0.089 


0.156 


1 


1 


47 


0.125 


0.063 


0.063 


1 


1 


48 


0.109 


0.082 


0.082 


1 


1 


49 


0.041 


0.189 


0.135 


0 


1 


50 


0.126 


0.058 


0.049 


1 


1 



G. PAIRED COMPARISONS 
1. Development 

Assume a comnarison of two teachers. Ti., Ti„\ under the constraint that the 

A *v 

voters must know each of them. Let 

N o 

N k,k’ = Z Yik yjk’ 
i=l 

where N Q = total number of ballots and y- k is an indicator function whose value 
is one if ballot i was acquainted with T k and zero otherwise. N k k > represents the 
number of voters who knew both T k and T k >. Note that N k k > is a symmetric matrix 
with zeroed diagonals. 

In order to determine the scores of T k and T k >, first determine Z k (k,k’) 
and Zj c (k’,k) w r here Z k (k,k’) represents the total score accumulated by T k from the 
population of votes that knew both Tj. and T k >. Thus, 

N 0 

Z k (k,k’) = Z yik^ik’ ( w l X ilk + w 2 X i2k + w 3 X i3k) 
i=l 

and 

So 

Z k (k’,k) = £ y ik yik’ ( w l X ilk' + w 2 X i2k' + w 3 X i3k) 
i=l 
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where, X- k is the number of j th place votes received by T k on the i th ballot. 

Then, 

S k (k,k’) = Z k (k,k’)/N k>k , 

and 

S k (k\k) = Z k (k’,k)/N k k > 

S k (k,k’) represents the score of T k under the constraint that the voters 
knew both T k and T k >. A similar interpretation can be made for S k (k’,k) and T k \ 

Now a fraction S k (k,k’)/S k (k’,k), is formed so that 

1) If S k (k,k’) > S k (k’,k) the value is greater than one. 

2) If S k (k,k’) < S k (k’,k) the value is less than one. 

3) If S k (k,k’) = S k (k’,k) the value is one. 

A reversal is noted if S k > S k ’ and 2) above occurs and a tie if 3) above occurs. By 
reversal is meant, the paired comparison fails to confirm the primary ordering of the 
two teachers. The significance of a reversal is enhanced the greater the value of N k k ?. 
2. Results 

A computer routine based on the above theory was run using the data and 
a comparison was made for T k , k=l, . . . , 7. Table IX presents the results of that 
run. In the table N is the number of voters who knew both T k and T k >. 

H. COMMENTS 

All individuals passing the dominance tests, zeros in both AD and CD, had the 
comment section of the ballots identifying them scrutinized and an abstract was made 
of the comments. In order to maintain complete objectivity of the selection committee 
all references to name or department were removed from the comment abstract. 

I. THE SELECTION 

Once all of the forementioned tests were performed and the results obtained, a 
set of statistics was prepared for the selection committee. This set of statistics 
included: 
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N 

38 

15 

10 

6 

4 

4 

N 

35 

15 

9 

6 

5 

4 

4 

3 



Table IX. PAIRED COMPARISONS 





Tl 






t 2 






T 3 


T k 


si/s k 


N 


T k 


S 2 / S k 


N 


T k 


S 3 / S K 


12 


129/142 < 1 


28 


9 


250/100 > 1 


4 


8 


025/250 < 1 


2 


140/160 < 1 


22 


18 


250/045 > 1 


4 


10 


025/200 < 1 


9 


190/160 > 1 


21 


17 


114/124 < 1 


3 


17 


033/033 = 1 


18 


333/017 > 1 


17 


19 


082/106 < 1 








13 


000/075 < 1 


15 


1 


160/140 >1 








17 


150/250 <- 1 


11 


12 


227/109 >1 












10 


6 


080/060 >1 












9 


8 


111/111 = 1 












9 


10 


100/022 > 1 












8 


13 


088/013 >1 












6 


4 


067/017 > 1 










t 4 






t 5 






t 6 


T k 


S 4 S k 


N 


T k 


S 5 / S k 


N 


T k 


V s k 


14 


191/203 < 1 


7 


9 


200/144 > 1 


20 


17 


095/050 > 1 


13 


133/033 > 1 


4 


8 


000/200 < 1 


10 


2 


060/080 < 1 


8 


122/067 > 1 


3 


17 


000/067 < 1 


10 


8 


690/160 > 1 




017/067 < 1 


3 

w/ 


18 


200/067 > 1 


5 


4 


160/000 > 1 


6 


120/160 < 1 








5 


9 


200/040 < 1 


18 


000/000 = 1 








5 


19 


080/000 > 1 


20 


100/000 > 1 








4 


13 


050/000 > 1 


17 


000/000 = 1 








4 


20 


125/000 > 1 












3 


19 


000/000 = 1 












3 


18 


033/000 > 1 



T 7 



N 


T k 


S 7/S k 


27 


10 


078/111 < 1 


19 


19 


221/079 > 1 


17 


11 


194/041 > 1 


15 


18 


193/027 > 1 


6 


14 


067/100 < 1 


5 


16 


080/080 = 1 


4 


17 


000/050 < 1 


3 


8 


200/000 > 1 


3 


13 


000/133 < 1 
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1) The results from the score calculations 

2) The results of the Dominance Tests 

3) The results of the paired comparison test 

4) The comment abstract. 

After deliberating for some time on the data, a decision was made for T£. That 
individual was identified to the selection committee as Professor E. C. Haderlie of the 
Department of Oceanography. 
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IV. FURTHER ANALYSES 



A. BACKGROUND AND SUMMARY 

It was observed last year by Geary [ref. 3] that the distribution of scores could 

be represented by the mixed density 

f s (s) = q for s = 0 

= n\e‘^- s for s > 0 

where p + q = 1 andX > 0. The parameter values were q = .177 and A = 2.09 for 

a population of N = 249 scores. The following questions were raised: 

1. Does the above distribution type hold again this year? If so, have the 
parameters remained the same? 

2. Is the distribution of scores the same for teachers who taught the quarter 
of the balloting and those who did not? If not, do the teachers who taught 
score better? 

3. Are the teacher rankings produced this year in concordance with those 
produced last year? 

These questions are addressed in subsections B, C, and D. 

In summary, 

A \ 

1. The mixed exponential distribution still holds qith q = .114, A = 2.41 
and N = 290. The direction of the parameter changes was noteworthy in 
that it supported the hypothesis that the award was encouraging better 
teaching on a broad scale. The proportion of zero scores had decreased 
and the positive scores had smaller variability. 

2. The segregating of teachers into two groups, those who taught in the 
quarter of the balloting and those who did not, resulted in score distri- 
butions which were still of the exponential type, but the parameterization 
changed. It was believed that those who did not teach would have lower 
scores, but the results were not conclusive. This is discussed in subsection C. 

3. The Kendall Concordance Coefficient was computed. The value of 0.77 
was mildly high, significantly above zero, but also significantly less than 
one. 
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B. SCORE DISTRIBUTION STUDY 

Figure 6 is a histogram of S k , the score of teacher k, s > 0, for FY ’71. Note 
the reverse J-shape which indicates there is a good chance for the distribution of 
the scores to be exponential. The data for FY ’70 had a similar histogram and 
Lieutenent Robert Geary [ref. 3] showed that the data was exponential with a 
density function 

fg(s) = 2.09 e' 2 - 09s , s > 0. 

A similar assumption was made this year and the maximum likelihhod estimation 
procedure for the parameter X was used. This results in 



X = 



\ 



where x is the mean of the sample of positive scores and is the parameter used in 
the exponential density function. Recall from Barr [ref. 1] that the probability 



density x unction of cm exp 



v> on tial function is 



fjr(x) = e'^ 04 , x > 0 



The log of the likelihood equation is 

n lnX - XXujfj 

where U: is the cell midpoint and f; is the cell frequency. Setting the derivative of 
the above with respect to X equal to zero yields 



u i f j k 

J=1 

thus 

\ 1 n 

X- ^ £ u i 'j- 

J=1 



Tables X through XVII show the data breakdown of the FY 70 and FY 71 data. 

A. 

Table XVIII shows the A values. 
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Figure 6. HISTOGRAM OF 



Table X. CHI SQUARE TEST DATA FY 70 ALL 



j 


k 


U i 


F i 


u i F i 


P i 


NPj 


Fj/NPj 


1 


1 


.05 


42 


2.10 


.1894 


38.83 


45.43 


2 


2 


.15 


34 


5.10 


.1601 


32.82 


35.22 


3 


3 


.25 


23 


5.75 


.1232 


25.26 


20.94 


4 


4 


.35 


27 


9.45 


.0999 


20.48 


35.60 


5 


5 


.45 


13 


5.85 


.0844 


17.30 


9.77 


6 


6 


.55 


13 


7.15 


.0650 


13.33 


12.68 


7 


7 


.65 


5 


3.25 


.0526 


10.78 


2.32 


8 


8 


.75 


11 


8.25 


.0427 


8.75 


13.83 


9 


9 


.85 


7 


5.95 


.0361 


7.40 


6.62 


10 


10 


.95 


7 


6.65 


.0278 


5.70 


8.60 


11 




1.05 


1 


1.05 








12 


11 


1.15 


3 


3.45 


.0415 


8.51 


1.88 


13 


12 


1.25 


5 


6.25 


.0146 


2.99 


8.36 


14 




1.35 


1 


1.35 








15 




1.45 


1 


1.45 








16 




1.55 


1 


1.55 








17 


13 


1.65 


2 


3.30 


.0359 


7.36 


3.40 


18 


14 


1.75 


4 


7.00 


.0051 


1.05 


15.24 


19 




1 .85 


0 


0.00 






1 


20 




1.95 


0 


0.00 








21 




2.05 


1 


2.05 








22 




2.15 


0 


0.00 








23 




2.25 


1 


2.25 








24 


15 


2.35 


3 


7.05 


.0217 


4.45 


5.62 



N=205 ZUjfj=96.25 



x = 1 Z Usf: = .4695 
n j=1 1 1 

A 

X= i= 2.13 
x 

*15-2, .05 = Z *V NP k ' N = 20,49 
k=l 
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Table XI. CHI SQUARE TEST DATA FY 70, Q3, TEACHING 



j 


k 


u i 


F i 


u i F i 


P i 


NPj 


Fj/NPj 


1 


1 


.05 


31 


1.55 


.1 894 


28.98 


33.16 


2 


2 


.15 


23 


3.75 


.1536 


23.50 


26.60 


3 


3 


.25 


18 


4.50 


.1297 


19.84 


16.33 


4 


4 


.35 


22 


7.70 


.0999 


15.28 


31.68 


5 


5 


.45 


10 


4.50 


.0809 


12.38 


8.08 


6 


6 


.55 


5 


2.75 


.0657 


10.05 


2.46 


7 


7 


.65 


4 


2.60 


.0532 


8.14 


1.97 


8 


8 


.75 


7 


5.25 


.0449 


6.87 


7.13 


9 


9 


.85 


7 


5.95 


.0347 


5.31 


9.23 


10 


10 


.95 


7 


6.65 


.0280 


4.28 


11.45 


11 




1.05 


1 


1.05 








12 




1.15 


1 


1.15 








13 


11 


1.25 


5 


6.25 


.0567 


8.68 


5.64 


14 




1.35 


1 


1.35 








15 




1.45 


0 


0.00 








16 




1.55 


1 


1.55 








17 




1.65 


1 


1.65 








18 


12 


1.75 


4 


7.00 


.0414 


6.33 


7.74 


19 




1.35 


0 


0.00 








20 




1.95 


0 


0.00 








21 




2.05 


0 


0.00 








22 




2.15 


0 


0.00 








23 




2.25 


1 


2.25 








24 


13 


2.35 


2 


4.70 


• .0219 


3.35 


2.69 



N=153 ZUjfj=72.15 



x = 1 I U:f: = .47156 
n j=1 1 1 



X= 1= 2.12 



1 3 

X?3-2 = Z F k> P k ' N = 1115 
k=1 
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Table XII. CHI SQUARE TEST DATA FY 70, Q3, NOT TEACHING 




N=52 ZUjfj=24.1 



x = 1 Z Ujfs = .46346 
n j=1 1 * 




2.16 



2 

X 7-2,.05 



7 2 

Z F k /NP k - N = 7.82 
k=1 
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Table XIII. CHI SQUARE TEST DATA FY 70, Q4, TEACHING 



U; 



F: 



U : F; 



P: 



NP: 



Fj/NPj 



1 


1 


.05 


30 


1.50 


.1975 


28.44 


31.65 


2 


2 


.15 


26 


3.90 


.1585 


22.82 


29.62 


3 


3 


.25 


13 


3.25 


.1323 


19.05 


8.87 


4 


4 


.35 


21 


7.35 


.1010 


14.54 


30.33 


5 


5 


.45 


9 


4.05 


.0811 


11.68 


6.93 


6 


6 


.55 


11 


6.05 


.0651 


9.37 


12.91 


7 


7 


.65 


5 


3.25 


.0523 


7.53 


3.32 


R 


8 


.75 


5 


3.75 


.0436 


6.28 


3.98 


9. 


9 


.85 


5 


4.25 


.0333 


4.80 


5.21 


10 


10 


.95 


5 


4.75 


.0267 


3.84 


6.51 


11 




1.05 


1 


1.05 








12 




1.15 


2 


2.30 








13 


11 


1.25 


3 


3.75 


.0530 


7.63 


4.72 


14 




1.35 


0 


0.00 








15 




1.45 


1 


1.45 








16 




1.55 


0 


0.00 








17 




1.65 


1 


1.65 








18 


12 


1.75 


2 


3.50 


.0373 


5.37 


O QR 

w | 


19 




1.85 


0 


0.00 








20 




1.95 


0 


0.00 








21 




2.05 


1 


2.05 








22 




2.15 


0 


0.00 








23 




2.25 


1 


2.25 








24 


13 


2.35 


2 


4.70 


.0183 


2.64 


6.06 



) ) 



=64.8 



i 24 

x = i I 

n j=1 



U j F i = 



.450 



A i 

A= = = 2.22 
x 

X?3.2,.05 ' Z F ? NP k ' N ' 9 ,0 
k=1 
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Table XIV. CHI SQUARE TEST DATA FY 70, Q4, NOT TEACHING 




x 



24 
n j=1 



UjFj 



= .5156 




1.94 



2 

x 9-2,.05 



I F ^/NP k • N = 3.75 
k=1 
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Table XV. CHI SQUARE TEST DATA FY 71, ALL 



U: 



U; F: 



NP: 



Fj/N Pj 



1 


1 


.05 


53 


2.65 


.2212 


56.85 


49.41 


2 


2 


.15 


51 


7.65 


.1600 


41.12 


63.25 


3 


3 


.25 


31 


7.75 


.1320 


33.92 


28.33 


4 


4 


.35 


27 


9.45 


.1039 


26.70 


27.70 


5 


5 


.45 


18 


8.10 


.0847 


21.77 


14.88 


6 


6 


.55 


20 


11.00 


.0636 


16.35 


24.46 


7 


7 


.65 


15 


9.75 


.0501 


12.88 


1 7.47 


8 


8 


.75 


5 


3.75 


.0393 


10.10 


2.48 


9 


9 


.85 


4 


3.40 


.0310 


7.97 


2.01 


10 


10 


.95 


7 


6.65 


.0244 


6.27 


7.81 


11 


11 


1.05 


6 


6.30 


.0191 


4.91 


7.33 


12 




1.15 


1 


1.15 








13 


12 


1.25 


4 


5.00 


.0260 


6.68 


3.74 


14 




1.35 


0 


0.00 








15 


13 


1.45 


4 


5.80 


.0179 


4.60 


3.48 


16 




1.55 


3 


4.65 








17 


14 


1.65 


3 


4.95 


.0104 


2.67 


13.48 


18 




1.75 


1 


1.75 








19 




1.85 


2 


2.70 








20 




1.95 


0 


0.00 








21 




2.05 


1 


2.05 








22 


14 


2.15 


1 


2.15 


.0164 


4.21 


5.94 



N=257 ZUjFj=1 06.65 



i 22 

x = - E 



n 



i=i 



UjFj 



= .4149 



A 



A= = = 2.41 
x 



2 

X 1 5-2, .05 



15 2 

E F^/NP k -N = 14.38 
k=1 
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Table XVI. CHI SQUARE TEST DATA FY 71, Q3, TEACHING 



P; 



NP; 



1 


K 


u j 


j 


) J 


1 


1 


J J 


1 


1 


.05 


40 


2.00 


.2055 


38.22 


41.86 


2 


2 


.15 


29 


4.35 


.1569 


29.18 


28.82 


3 


3 


.25 


20 


5.00 


.1310 


24.37 


16.41 


4 


4 


.35 


22 


7.70 


.1041 


19.36 


25.00 


5 


5 


.45 


15 


6.75 


.0827 


15.38 


14.63 


6 


6 


.55 


14 


7.70 


.0631 


11.74 


16.70 


7 


7 


.65 


13 


8.45 


.0528 


9.82 


17.21 


8 


8 


.75 


5 


3.75 


.0419 


7.79 


3.21 


9 


9 


.85 


4 


3.40 


.0320 


5.95 


2.69 


10 


10 


.95 


5 


4.75 


.0267 


4.97 


5.03 


11 


11 


1.05 


4 


4.20 


.0212 


3.94 


4.06 


12 




1.15 


1 


1.15 








13 


12 


1.25 


3 


3.75 


.0298 


5.54 


2.89 


14 




1.35 


0 


0.00 








15 




1.45 


3 


4.35 








16 




1.55 


1 


1.55 








17 


13 


1.65 


2 


3.30 


.0312 


5.80 


6.21 


18 




1 .75 


1 


1.75 








19 




1.85 


2 


3.70 








20 




1.95 


0 


0.00 








21 




2.05 


1 


2.05 








22 


14 


2.15 


1 


2.15 


.0211 


3.92 


6.38 



N=1 86 



ZUjFj=81.8 



i 22 

X = 1 Z U: F: = .439 
"i=1 M 



\ 1 

A= L = 2.27 
x 



x 14-2, .05 = I Fk' NP k ‘ N = 509 
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Table XVII. CHI SQUARE TEST DATA FY 71, Q3, NOT TEACHING 




= - z UjFj = .364 
n j=1 11 




2.75 



2 

X 7-2 ,.05 



7 2 

I F k /N p k - N = 13.41 
k=1 
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Table XVIII. VALUES 



Group 




FY 70 


FY 71 


Q3 


Q4 


Q3 


All Teachers 


2.13 


2.13 


2.41 


Those Teaching 


2.12 


2.22 


2.27 


Those Not Teaching 


2.16 


1.94 


2.75 



A Chi-Square Goodness-of-Fit Test was used to test the hypothesis that the 

density functions determined using the parameters above were valid representations of 

their respective data. The null and alternate hypotheses for the above functions were 

A \ 

H 0 : f s (s) = Ae-As, s> 0 



H-j^: Hq false, 

A 

where A is the value for the parameter of the density function, n is the number of 
positive scores, I 1? I 2 , . . . I r represents the scoring intervals, pj = P(S€lj) under Hq, 
npj is the expected number of entries in Ij under H 0 and Fj is the number of observed 
entries in Ij. 



Define 




(Fj - NPj)' 



NP: 





- N 



where N = Z F-. As n gets large, z approaches the Chi-square distribution function 

j J 

with r-2 degrees of freedom 
Thus let 

P l X r-2,(l-Q) > ° bs Ts|Hq] = P 



Compare p withQand if 

1. p < CL then reject Hq, 

2. p > CL then accept Hq. 

The values determined for p were : 
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Table XIX. p VALUES 



All Teachers 
Those Teaching 
Those Not Teaching 



Group 



FY 70 

Q3 Q4 

.084 .084 

.403 .613 

.168 .800 



FY 71 
Q3 
.350 
.995 
.030 



Thus, it was concluded from the p values that Hq could not be rejected for 
reasonable levels of significance. 

C. TEACHING EFFECT 

The author felt a reasonable question arose during the data manipulation con- 
cerning the effect of teaching during the voting quarter on a teacher’s ranking. It 
seemed logical that a voter would be more likely to rank a teacher who was currently 
instructing than one who was off for the quarter. This assumption was based on the 
conjecture that a person is more likely to hold a stronger opinion of a teacher he 
is associating with at the time than one he has had in the past. 

To test this hypothesis it would be best to remove all non-student ballots, but 
this was not possible due to time constraints. The faculty from the Navy Management 
Systems Center were removed from the data for they do not teach by quarters. 

Care had to be exercised for during the FY 70 period the balloting was done at 
the start of Quarter 4, Q4. Since it was felt that strong opinions could not be 
formed during the first two weeks of a quarter, the data for Q3 was also gathered. 

The FY 71 period did not present the problem for the balloting was done near the 
end of Q3 for that year. 

Using the model 



f s (s) = q, s = 0 

= pXe‘^- s , s > 0 



it is desired to test 
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against 

q 2 > qi and X 2 > Xi 

The author knows of no standard practice for this statistical problem. The following 
ad hoc discussion will shed some light on the question. 

Reject H 0 if q 2 is sufficiently larger than q x and A 2 is sufficiently 
larger that Aj. 

However, these estimates are not statistically independent and joint probability 
statements concerning them are difficult to calculate. In lieu of this, the pairs X^,X 2 

A A 

are examined separately from q^, q 2 . 

Table XVIII showed thatX 2 > A 1 in Quarter 3 of both years, butX2 < X x in 
Q4 of FY 70. Thus, the direction of the shift was consonant with the discussion at 
the beginning of this section. To determine the significance of the change, some 
distribution theory is required. 

Since a 

X2 n 2 X 1 

Xj ’ "1 X 2 

where X| is the sum of positive scores for group i and nj is the number of positive scores; 
and since a positive score has an exponential distribution, it follows that 

Xj-~P(nj,X), 2Xxj'-'-' X^ 2n p 

and it follows that 



X 2 

£ 



'2n^,2n 2 ' 



The p values are listed in Table XX. 

Table XX. p VALUES FOR X 2 /X«| 



A A 



Year 


Quarter 


Test Statistic 


d.f. 


1970 


3 


2.16/2.12= 1.019 


104/306 


1970 


4 


1.94/2.22 = 0.874 


122/288 


1971 


3 


2.75/2.27 = 1.211 


142/372 



Approx, p Value 
.40 
.75 
.05 
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Turning to and c^, the normal approximation will be used in the two sample 
case. Let 

h^ = percentage of zeroes of those professors teaching during the observed 
quarter. 

1>2 = percentage of zeroes of those professors not teaching during the observed 
quarter. 

Then 

n l h l + n 2 h 2 
h 

n l +n 2 

where h is the pooled percentage of all teachers, nj^ is the number of professors teaching 
during the observed quarter, and n 2 is the number of professors not teaching during the 
observed quarter. Now let 

<T = v/h(l-h)(l/ ni + l/n 2 ) , 

then from Brownlee [ref. 2] 



(h 2 - l/2n 2 ) - (h-L + l/2 ni ) 

A 

CT 



N(0,1) 



using this formula, solving for p values yields 



Table XXI. p VALUES FOR q TEST 



Observed Quarter p Value 

Q3.FY70 .14 

Q4, FY 70 .79 

Q3, FY 71 .09 



Again, the direction of the change supported the discussions at the beginning of 
this section, but the results were not significant using this as hoc approach. A sharper 
analysis may or may not give more significant support to this phenomenon. Such an 
analysis would include 

1. Removal of the non-student ballots 

A a 

2. Correct joint probability statements concerning A^, A 2 ’ an d 92- 
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D. CONCORDANCE ESTIMATION 



Suppose that X = (X a , X* 3 ) has a bivariate cumulative distribution function 

. a a 

F(x a ,x D ), then the vectors x^ and X£ are said to be concordant if xj - X£ and 

x^ - X2 have the same sign. Further, using Kendall’s [ref. 5] Coefficient of Concordance 

the commonality of two sets of data can be determined. If the test is performed and 

the coefficient of concordance is near one, the two sets of data show similar ranking. 

The further the coefficient is from one, near zero, the worse the relationship. Thus 

this test can be used to determine if a teacher scoring high one year is likely to score 

high the next. Also, it can be determined if the students rank the teachers the same 

each year. 

Kendall, [ref. 5], defines the relationship as 

, v 12S 

m 2 (n 3 - n) - 12m ZTj 

where S is the sum of squares of actual deviations, n is the number of rankings, m 
is the number of raters, Tj = Z^(t 3 - t), and t indicate ties. 

Table XXII is a 2 x 222 matrix of rankings determined from the scores.. The 
first row represents the ranking of teachers in FY 70 and the second row represents 
the ranking of the teachers in FY 71. The choice of 222 teachers was made so that 
the population would include only professors eligible both years. 

From this table the values of the equation take the form 



W = 



12(2743884,46) 

(2) 2 [(222) 3 - 222] - 12(2) (47838) 



yielding 

W = .77264 

Kendall also expressed a method to determine the significance of the Concordance 
Coefficient. Let 



2 

X r 



m(n-l) W = 



12S 

mn(n+l) 



where r = n-1, yielding 
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Table XXII. CONCORDANCE MATRIX 



FY 70 


1 


2 


3 


4 


FY 71 


8 


2 


5 


12 


Total 


9 


4 


8 


16 


FY 70 


12 


13 


14 


15 


FY 71 


19 


194 


25 


41 


Total 


31 


207 


39 


56 


FY 70 


23 


24 


25 


26 


FY 71 


28 


130 


13 


34 


Total 


51 


154 


38 


60 


FY 70 


34 


35 


36 


37 


FY 71 


74 


20 


29 


21 


Total 


108 


55 


65 


58 


FY 70 


45 


46 


47 


48 


FY 71 


177 


58 


111 


83 


Total 


222 


104 


158 


131 


FY 70 


56 


57 


58 


59 


FY 71 


100 


96 


59 


39 


Total 


156 


153 


117 


98 


FY 70 


67 


68 


68 


70 


FY 71 


131 


129 


57 


37 


Total 


198 


197 


125 


107 


FY 70 


78 


79 


80 


81 


FY 71 


122 


46 


123 


135 


Total 


200 


125 


203 


216 


FY 70 


89 


90 


91 


92 


FY 71 


133 


84 


89 


60 


Total 


222 


174 


180 


152 


FY 70 


100 


101 


102 


103 


FY 71 


154 


134 


82 


116 


Total 


254 


225 


184 


219 


FY 70 


111 


112 


113 


114 


FY 71 


77 


129 


94 


71 


Total 


188 


241 


207 


185 



6 


7 


8 


9 


10 


11 


4 


37 


26 


18 


7 


16 


10 


34 


34 


27 


17 


27 


17 


18 


19 


20 


21 


22 


11 


9 


79 


136 


33 


40 


28 


27 


98 


156 


54 


62 


28 


29 


30 


31 


32 


33 


10 


51 


114 


24 


6 


181 


38 


80 


144 


55 


38 


214 


39 


40 


41 


42 


43 


44 


52 


75 


26 


23 


176 


48 


91 


115 


67 


65 


219 


92 


50 


51 


52 


53 


54 


55 


62 


17 


189 


15 


65 


61 


112 


68 


241 


68 


119 


116 


61 


62 


63 


64 


65 


66 


42 


134 


72 


70 

/ W 


108 


137 


103 


196 


135 


134 


173 


203 


72 


73 


74 


75 


76 


77 


121 


143 


55 


56 


98 


32 


193 


216 


129 


131 


174 


109 


83 


84 


85 


86 


87 


88 


49 


83 


109 


131 


65 


88 


132 


167 


194 


217 


152 


176 


94 


95 


96 


97 


98 


99 


87 


171 


73 


54 


126 


138 


181 


266 


169 


151 


224 


237 


104 


106 


107 


108 


109 


110 


166 


80 


125 


113 


197 


146 


270 


186 


232 


221 


306 


256 


116 


116 


118 


119 


120 


121 


127 


91 


147 


106 


148 


128 


243 


207 


265 


225 


268 


249 



5 

1 

6 

16 

30 

46 

27 

162 

189 

38 

14 

52 

49 

142 

191 

60 

31 

91 

71 

35 

106 

82 

33 

115 

93 

150 

243 

104 

64 

168 

115 

88 

203 
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Table XXII. CONCORDANCE MATRIX (concluded) 



FY 70 


122 


123 


123 


123 


FY 71 


105 


204 


145 


22 


Totals 


227 


327 


268 


145 


FY 70 


133 


134 


135 


136 


FY 71 


193 


45 


188 


44 


Total 


326 


179 


323 


180 


FY 70 


144 


145 


146 


147 


FY 71 


170 


159 


102 


175 


Total 


314 


304 


248 


322 


FY 70 


155 


156 


157 


158 


FY 71 


117 


166 


206 


172 


Total 


272 


322 


363 


330 


FY 70 


166 


167 


168 


169 


FY 71 


99 


101 


152 


161 


Total 


265 


268 


320 


330 


FY 70 


1 - 7-7 
1 / / 


178 


179 


180 


FY 71 


190 


191 


206 


199 


Total 


367 


369 


385 


379 


FY 70 


188 


188 


188 


188 


FY 71 


143 


160 


184 


153 


Total 


331 


348 


372 


341 


FY 70 


188 


188 


188 


188 


FY 71 


156 


206 


155 


169 


Total 


344 


394 


343 


357 


FY 70 


188 


188 


188 


188 


FY 71 


185 


206 


206 


141 


Total 


373 


394 


394 


329 


FY 70 


188 


188 






FY 71 


206 


206 






Total 


394 


394 







127 


128 


129 


129 


131 


132 


93 


65 


165 


90 


149 


104 


220 


193 


294 


219 


280 


236 


138 


139 


140 


141 


142 


143 


206 


201 


50 


95 


198 


183 


344 


340 


190 


236 


340 


326 


149 


150 


151 


152 


153 


154 


173 


178 


140 


139 


63 


119 


322 


328 


291 


291 


216 


273 


160 


161 


162 


163 


164 


165 


115 


166 


158 


181 


202 


203 


275 


327 


320 


344 


366 


368 


171 


172 


173 


174 


175 


176 


47 


180 


107 


112 


92 


206 


218 


352 


280 


286 


267 


382 


182 


183 


184 


185 


i 

• w w 


187 


200 


76 


195 


164 


118 


206 


382 


259 


379 


349 


304 


393 


188 


188 


188 


888 


188 


188 


151 


186 


120 


157 


206 


206 


339 


374 


308 


345 


394 


394 


188 


188 


188 


188 


188 


188 


187 


206 


205 


196 


163 


182 


375 


394 


393 


384 


351 


370 


188 


188 


188 


188 


188 


188 


110 


206 


97 


36 


179 


3 


29S> 


394 


265 


224 


367 


191 



126 

53 

179 

137 

68 

205 

148 

69 

217 

159 

192 

351 

170 

174 

344 

181 

206 

387 

188 

206 

394 

188 

206 

394 

188 

85 

273 
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2 

x r = 341.507 



/ 2 

For values of r greater then 30, the quantity V 2x r may be taken to be 
distributed normally about the mean, \/2r-l . Thus 

n/~ 2? , — x N( \T~2tA., 1) 



or 



or 



So 



or 



2x 



221 



N(/l41, 1) 



2x 



221 - 21 N(0,1) 



P(j2x% 21 - 21 > z) = .05 



% x 221 ' 21 > 1 - 65 



ZZ> *221 > 25S - 51 
This specifies the critical region. Since 

2 

341.507 > *221 = 256 - 51 

it was concluded that the resulting W = .77 is substantially significant. This 
indicated that the rankings of teachers was not random and that a teacher’s 
ranking was indicative, at least to a small extent, of the way the voting population 
as a whole felt about him. 
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V. CONCLUSIONS 



The ballot format changes discussed in Section I are recommended. It is also 
recommended that a year to year update of accumulated balloting statistics be made 
as part of the automatic data processing. These include specifically 

1. Tabulation of voter characteristics. 

2. Computation of the mixed exponential parameters. 

3. Concordance estimates be determined. 

The study of the effect on the scores of teaching in the quarter of the balloting 
was inconclusive for the reasons given in Section IV. It is recommended that plans 
be made so that this question can be firmly resolved in the future. 
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APPENDIX A 
SAMPLE BALLOT 

SUPERINTENDENT'S AWARD FOR EXCELLENCE IN TEACHING 



In order to augment the existing incentives for teaching of high quality, 
the Superintendent has established an award called the Superintendent's Award for 
Excellence in Teaching. The judgments of those who have participated in the educa- 
tional processes at the Naval Postgraduate School are of primary importance. This 
poll is being conducted to assess the collective opinion of students, faculty, staff, 
and selected alumni. After accumulating and processing the voting results, the 
Selection Committee will recommend an Awardee to the Superintendent. 

In view of the many measures of teaching excellence, there is no intent 
to fix the rationale by which each voter determines his nominations. However, in 
order to assist those voters who desire something more definite as a guide to their 
thinking, the following summary of some aspects of teaching excellence is offered as 
worthy of consideration: "The phrase, 'excellence in teaching,' refers to that com- 

plex of personal and professional qualities and actions on the part of the teacher 
which (a) make themselves felt primarily at the interface of personal contact between 
student and teacher; (b) help transmute the student's encounters with his subject 
matter into insight, enlightenment and love of learning; (c) elicit from the student 
responses in thought, feeling and action which enhance his capacity for self education 
and (d) manifest themselves in an effective individual style which authentically re- 
flects the teacher's own unique personality, experience, character, and convictions." 

A meaningful selection of the Award recipient is heavily dependent upon 
the receipt of nominations from a large portion of the eligible "voters". Your 
response is therefore earnestly requested. Please complete the enclosed ballot in 
accordance with the attached instructions. The results of the polling, other than 
the Award recipient, will be treated as privileged information. 

BALLOTING PROCEDURE: 

Step 0. Complete the information requested at the top (Part A) of the ballot. 

These items are for purposes of statistical analyses only. 

Step 1. On the list of eligible faculty (Part C), encircle the four-digit 
identification number at the left of all those with whom you are 
sufficiently acquainted to make a judgment. Use your own guidelines. 

For purposes of establishing a valid population base, it is important 
to encircle all with whom you are acquainted. It is the judgment of 
the Selection Committee that you should be acquainted with at least 
five eligible faculty members in order to cast a meaningful ballot. 
Accordingly, if you have circled five or more numbers on the ballot, 
proceed to Step 2. Otherwise, please return your ballot as directed 
in Step 4. 

Step 2. From the subset of faculty you have indicated in Step 1, select from 
one (1) to three (3) nominees. Indicate your preferences in order 
by placing the four-digit identification numbers of your nominees 
for first, second and third choices in the appropriate spaces pro- 
vided in Part B. 

Step 3. You are invited to furnish a short statement in support of your 

primary nomination. Space is provided for this at the end (Part D) 
of the ballot. 

Step 4. Place your completed ballot in the enclosed return envelope, sign 
the envelope and return within ten days. Ballots will be separated 
from the envelopes. 
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dMLLU I 



Part A - Statistical (encircle code at left) 

Voter Category 

Military 



S Student 
F Faculty 
A Alumnus 

C Curricular Officer 
Civilian 



Rank 


Branch of Service 


Curricular Area 




Rank 




Department 


01 


N 


Navy 


30 


Ops Analysis 


IR 


Instructor 


AE 


Aeronautics 


02 


M 


Marine Corps 


31 


Aero Eng 


AT 


Assist Prof 


A0 


Aviation Safety 


03 


A 


Army 


32 


Elec & Comm Eng 


AC 


Assoc Prof 


MN 


Bus Ad & Econ 


04 


C 


Coast Guard 


33 


Ordnance 


PR 


Professor 


EE 


Elec Eng 


05 


L 


Air Force 


34 


Naval Eng 






GH 


Govt & Humanities 


06 


F 


Foreign 


35 


Env Sci 






MC 


Mat Sc & Chem 


07 






36 


Mange & Comptr Sci 






MA 


Math 








37 


Eng Sci 






ME 


Mech Eng 








38 


Baccalaureate 






MR 


Meteorology 








39 


Def Management 






OC 


Oceanography 
















0A 


Ops Analysis 
















PH 


Physics 
















NS 


Navy Mange Sys Center 


Part 


B - 


Nominations (use 


the 


four-digit identification number) 










First choice 




Second choice 




Third choice 







Part C - List of Eligible Faculty (all faculty engaged in teaching in the academic year (1970-71 
except department chairmen and members of che selection committee) 



1014 Bomberger, R. B. 
1025 Bradbury, C. M. 

1036 Fletcher, J. L. 

1047 Wible , L. C. 
Department of Aeronautics 


1058 


Ball, R. E 


. 


1059 


B°r ns tt , J 


. A. J. 


1070 


Biblarz, 0 


. 


1081 


Collins, D 


. J. 


1092 


Gawain, T. 


H. 


1108 


Haupt, U. 




1119 


Kahr , C. H 


. 


1120 


Layton, D. 


M. 


1131 


Lindsey, G 


. H. 


1142 


Miller, J. 


A. 


1153 


Netzer, D. 


W. 


1164 


Platzer, M 


. F. 


1175 


Redlin, M. 


H. 


1186 


Schmidt, L 


. V. 


1197 


Vavra, M. , 


H. 


1203 


Zucker, R. 


D. 


Depar 


tment of Bu 


si ness 


Administration and Economics 



1214 


Carrick, P 


. M. 


1225 


Church, W. 


H. 


1236 


Cowie, J. 1 


B. 


1247 


Darbyshi re 


, L. 


1258 


Ei senhardt 


• P. 


1269 


Elster, R. 


S. 


1270 


Fremgen, J 


. M. 


1281 


Ganz, J. G 




1292 


Githens, W 


. H. 


1308 


Grainger, 


T. L 



1319 Hoverland, H. A. 

1320 Hynes, J. P. 

1331 Jolly, J. A. 

1 342 Kushnick, S. A. 

1353 Lande, R. S. 

1364 Lane, H. L. 

1375 Lee, M. 

1386 McDonald, J. H. 

1397 Moan, F. E. 

1403 Noorzoy, S. 

1414 Smith, J. 

1425 Steckler, M. J. 

1436 Vigen, J. W. 

1447 Waibel, R. 

1458 Wegener, W. H. 

1469 Wiskoff, M. F. 

1470 Womer, N. K. 

Department of Electrical 1803 
Engi neeri ng 

1481 Adler, R. W. 

1492 Badger, R. 

1508 Bauer, W. M. 

1519 Baycura, 0. M. 

1 520 Borst, F. W. , Jr . 

1531 Bouldry, J. M. 

1542 Breida, S. 

1 553 Campbel 1 , J . D. 

1564 Chan, S. 

1575 Chaney, J. G. 

1586 Cooper, P. E. 

1597 Cotton, M. L. 

1603 De Laura, R. D. 

1614 Demetry, J. S. 

1625 Ewing, G. D. 



Gardner, E. M. 1953 

Geist, J . M. 1 964 

Gerba , A. , Jr . 1 975 

Houston, R. K. 1986 

Kirk, D. E. 1997 

Klamm, C. F. , Jr. 2006 
Marmont, G. H. 2017 

Miller', R. L. 

Murray, R. P. 

Myers, G. A. 

Myers, H. L. 

Oler, C. B. 

Panholzer, R. 

Rahe, G. A. 

Rothauge, C. H. 

Sackman, G. L. 
Sheingold, A. 

Smith, W. C. 

1814 Spaugy, D. A. 

1825 Stentz, D. A. 

1836 Strum, R. D. 

1847 Terman, F. W. 

1858 Thaler, G. J. 

1869 Titus, H. A. 

1870 Turner, J . B . , Jr. 

1881 Ward, J. R. 

1892 Wilcox, M. L. 

Department of Government 2190 
and Humani ties 



1636 

1647 

1658 

1669 

1670 
1681 
1692 
1 708 

1719 

1720 
1731 
1742 
1753 
1764 
1775 
1786 
1797 



Gottschalk, S. 
McAdams, J. M. 
O’Neil, E. F. 
Pearson, L. W. 
Smith, B. M. L. 
Stolfi, R. H. 
Teti, F. M. 

Department of Material 
Science and Cneimstry 



2028 

2039 

2040 
2051 
2062 
2073 
2084 
2095 
2101 
2112 
2123 



1908 Alexander, W. P. 

1919 Amos , J . W. , II 

1920 Bjarnason, L. L. 
1931 Boggess, W. C. 
1942 Gabel, B. B. 



Clark, J. R. 

Hell i well, R. W. 
Hering, C. A. 
Kinney, G. F. 
Leonesio, R. B. 
Reinhardt, R. A. 
Reynolds, M. F. 
Rowel 1 , C. F . 

Si nc lair, J. E. 
Tolies, W. M. 
Wilson, J. W. 
Department of Mathematics 
2134 Bender, A. P. 
Bleick, W. E. 
Bolles, R. C. 
Budway, J. J. 
Calabrese, P. G. 
Chewning, W. C. 
Comstock, C. 

2206 Davis, D. L. 

2217 Dixon, D. R. 

2228 Es tel 1 , R. J. 

2239 Faulkner, F. D. 

2240 Franke, R. 

2251 Giarratana, J. 



2145 

2156 

2167 

2178 

2189 
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2262 


Gibbons, G. D. 


Department of Meteorology 


2273 


Hartkemeier, H. P. 


2673 


Alberty, R. L. 


2284 


Hunt, R. W. 


2695 


Carrigan, R. C. 


2295 


Jayachandran , T. 


2684 


Davidson, K. L. 


2301 


Jennings, W. 


2701 


Duthie, W. D. 


2312 


Ki 1 dall , G. A. 


2712 


Elsberry, R. L. 
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